Skip to content

ci(repo): mock firebase for e2e tests#2821

Open
testableapple wants to merge 5 commits into
masterfrom
ci/e2e-mock-firebase
Open

ci(repo): mock firebase for e2e tests#2821
testableapple wants to merge 5 commits into
masterfrom
ci/e2e-mock-firebase

Conversation

@testableapple

@testableapple testableapple commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Resolve https://linear.app/stream/issue/FLU-602

What

Keeps the native Firebase SDK (Core / Crashlytics / Messaging) out of e2e test builds entirely — nothing is downloaded, compiled, linked, or initialized during e2e runs — and cleans up the iOS build caching in the e2e workflows based on measured data.

How the Firebase mock works

  • sample_app/e2e_stubs/ contains three pure-Dart no-op packages named firebase_core, firebase_crashlytics, and firebase_messaging, stubbing exactly the API surface sample_app uses (Firebase.initializeApp, FirebaseOptions, recordError/recordFlutterFatalError, the messaging streams/token getters, etc.). Permission requests report denied, tokens resolve to null, message streams never emit.
  • Because the stubs have no flutter.plugin section, the Flutter tool registers no plugins for them → no Firebase pods on iOS, no Firebase Gradle deps on Android.
  • sample_app/pubspec_overrides.e2e.yaml wires them in via dependency_overrides. The e2e workflows copy it over pubspec_overrides.yaml before melos bootstrap — melos merges its own managed overrides into the existing file and preserves the stub entries.
  • No app code or project.pbxproj changes are needed: the existing isE2eTestRun guards cover runtime behavior, and pod install regenerates the CocoaPods build phases on the runner to match the resolved plugin set.

Also in this PR

  • CocoaPods cache step moved after Bootstrap so its key hashes the resolved pubspec.lock (the lock is gitignored and doesn't exist before melos bootstrap — previously that part of the key was always empty and never rotated on dependency changes).
  • Removed the Xcode compilation cache (cache-xcode-cas). A verbose diagnostic run proved it works end-to-end on CI (1,734 replay cache hit remarks, 0 misses) but delivers only ~10% build-time benefit after normalizing for runner speed (the net is roughly zero on a good day and negative on rotation days) — cache replay is IO-bound and the non-cacheable tail (linking, script phases, kernel compile) dominates on the standard 4-vCPU runners — while costing ~12s restore per run, ~25s save on key rotation, and ~1 GB of cache quota. Recoverable from git history if we move to larger runners.

Validation

  • CI run with stubs active: pod install contains zero Firebase/Google/nanopb/Promises targets; tests green.
  • Locally: .flutter-plugins-dependencies has no firebase entries with stubs active, flutter analyze clean, and melos bootstrap preserves the stub overrides.
  • Local development is unaffected — the override file is only applied in CI.

Test (local)

Screenshot 2026-07-17 at 10 32 38

🤖 Generated with Claude Code

Summary by CodeRabbit

Summary by CodeRabbit

  • Bug Fixes
    • Improved iOS end-to-end reliability by using local no-op Firebase stubs instead of real Firebase services.
    • Adjusted iOS CI caching so CocoaPods cache keys are based on the resolved lockfile.
  • Tests
    • Added stub implementations for Firebase Core, Crashlytics, and Messaging for end-to-end test runs.
  • Chores
    • Simplified end-to-end workflows by removing the Xcode compilation cache and standardizing the E2E workflow naming.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a6ea87cc-ab4f-456f-be73-fb396620d755

📥 Commits

Reviewing files that changed from the base of the PR and between 325b532 and 924f27e.

📒 Files selected for processing (1)
  • .github/workflows/e2e_test.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/e2e_test.yml

📝 Walkthrough

Walkthrough

Changes

Firebase E2E builds now use local no-op stubs for Core, Crashlytics, and Messaging through pubspec overrides. Both iOS E2E workflows activate those overrides and run CocoaPods caching after dependency bootstrap. The Xcode CAS composite action and its workflow usage are removed.

Firebase E2E CI integration

Layer / File(s) Summary
Firebase stub package APIs
sample_app/e2e_stubs/*
Adds local Core, Crashlytics, and Messaging packages implementing the Firebase APIs used by the sample app with no-op behavior.
E2E dependency override wiring
sample_app/pubspec_overrides.e2e.yaml, .github/workflows/e2e_test.yml, .github/workflows/e2e_test_cron.yml
Redirects Firebase dependencies to local stubs and copies the E2E override file into place for iOS jobs.
Post-bootstrap iOS caching
.github/workflows/e2e_test.yml, .github/workflows/e2e_test_cron.yml
Moves CocoaPods caching after Bootstrap so cache keys use the resolved lockfile and removes Xcode CAS caching.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: xsahil03x, velikovpetar

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: mocking Firebase for e2e tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/e2e-mock-firebase

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@testableapple
testableapple force-pushed the ci/e2e-mock-firebase branch from b63b5a8 to 373de71 Compare July 16, 2026 16:44
@testableapple
testableapple marked this pull request as ready for review July 16, 2026 19:14
@testableapple
testableapple requested a review from xsahil03x July 16, 2026 19:14
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.28%. Comparing base (5f7f33c) to head (924f27e).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2821   +/-   ##
=======================================
  Coverage   71.28%   71.28%           
=======================================
  Files         430      430           
  Lines       26934    26934           
=======================================
  Hits        19199    19199           
  Misses       7735     7735           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant